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1 Introduction 


Constraint Satisfaction Problems (CSP) are among the most natural computational problems, and 
yet their computational complexity is not fully understood. In particular several works have studied 
the notion of Approximation Resistance, which loosely speaking means that the best polynomial-time 
approximation algorithm is simply the one that outputs a random assignment. Under Khot’s Unique 
Games Conjecture [16] much is known about this property. In particular Austrin and Mossell |3] 
showed if the UGC is true, then, for every predicate P : {0,1}^ —)• {0,1}, if there exists a pairwise 
independent distribution /r over P“^(l) (i.e., a distribution jjL such that for every i ^ j G [k], the 
marginal is the uniform distribution over {0,1}^), then P is approximation resistant. Austrin 
and Hastad |2] used this to establish (under the UGC) fairly tight bounds on the threshold at 
which a random predicate of a particular density becomes approximation resistant. However, there 
is no consensus whether the UGC is true. Assuming only P ^ NP, the best known bound is by 
Chan )10| who showed that a predicate is approximation resistant if it contains a distribution /r 
as above satisfying the additional condition that it is uniform over a subspace V C GF{2)^ . This 
algebraic structure is a fairly strong condition. In particular if we choose P : {0,1}*^ —)• {0,1} to be a 
random predicate conditioned on |P“^(1)| = t (where t G {1... 2^} is some parameter), then P will 
satisfy the hrst condition (supporting a pairwise independent distribution) with high probability as 
long as t > ck'^ for some constant c | 2 ] while it will not satisfy the second condition even for t as 
large as exp(/c/5) (see Observation A.l). 

Another line of work has been concerned with proving uneonditional lower bounds for these prob¬ 
lems on restricted families of algorithms. These works considered convex relaxations for CSPs, where 
we say that a CSP is approximation resistant for some relaxation TZ if there is an instance for which a 
random assignment is essentially optimal, but the relaxation value is 1 — o(l) (namely, the relaxation 
“thinks” that it’s possible to satisfy almost all constraints). Interestingly, the unconditional results 
match the conditional ones. That is, for certain weaker relaxations (namely, the Sherali-Adams 
linear programming hierarchy or Sherali-Adams augmented with the basic semidehnite program), 
there are unconditional results for the same predicates that were shown approximation-resistant 
under the UGC |9l[26l|T9]. (This is of course not a coincidence, as the UGC is intimately connected 
with some of these weaker relaxations | 21 ].l In contrast, for the stronger Sum of Squares (SOS) 
(also known as Lasserre) relaxation |24[|18ll20lll7j . the previously known results |15l [221125] utilized 
the same conditions as in Chan’s NP-hardness result (and in fact inspired Chan’s work). 

In this work we show that the pairwise independence condition suffices for lower bounds even for 
this stronger Sum-of-Squares hierarchy. This result is interesting in its own right and, based on past 
experience, could also be viewed as suggesting that it may be possible to improve the UGC-based 
results to results based on P 7 ^ NP. 


1.1 Our results 

Our results actually hold for a more general setting than showing approximation-resistance of predi¬ 
cates, and so to state them we need to introduce some notation. Roughly speaking, we show that for 
every k and an arbitrarily small e > 0, there exists a set X = {Ci, ..., Cm} of /c-tuples of literals (i.e. 
variables or their negations) over the variables xi,... ,Xn such that (1) for every assignment x to 
the variables, the induced distribution on { 0 , 1}*’ obtained by taking a random i G [m] and looking 
at the literals in Ci is e-close to the uniform distribution on { 0 ,1}^ but (2) for every pairwise inde¬ 
pendent distribution /i over {0,1}^, there is a relaxation-solution that “cheats” the H(n)-degree SOS 
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relaxation to think that there is a distribntion V over assignments (i.e. {0,1}^) snch that for every 
i G [m], the projection of T> to the literals in C* is distribnted according to /r. This immediately 
implies that predicates snpporting a pairwise independent distribntion are approximation-resistant 
for this relaxation. We now formally state onr resnlts: 

Definition 1.1 (Psendo-expectation). For every n and d, let denote the linear space of n-variate 
real polynomials of degree at most d. A linear operator E : —)• M is a degree-d pseudo-expectation 
operator if it satishes: 

Normalization E[l] = 1 where on the LHS 1 denotes the constant polynomial p snch that p{x) = 1. 
Positivity ]E[p^] > 0 for every p G 'P^/ 2 - 

For every polynomial p G V^, we say that E satishes the constraint {p = 0} if E[pg] = 0 for 
every q G 7^^deg(P)- 

The Snm-of-Sqnares hierarchy can be thonght of as optimizing over psendo-expectations; see the 
snrvey |8] and the references therein, as well as the lectnre notes |lj. For notational convenience, 
we will nse variables over {±1} instead of {0,1}. A literal is a fnnction / : {±1}*^ —)> {±1} snch 
that f{x) = Xi or f[x) = —Xi for some i. If C = (/i,..., fk) is a /c-tnple of literals then we denote 
by C{x) the tnple (/i(x),..., fk{x))- Onr main resnlt is the following: 

Theorem 1.2 (Main Resnlt). For every A; G N, e > 0 there exists 5 = 6{k) > 0 such that for every 
sufficiently large n G N there is a set X = {Ci,..., Cm} of k-tuples of literals over xi,... ,Xn such 
that 

1. For every x G {±1}"', the distribution {C(a:)} where C is chosen at random in X is within e 
statistical distance to the uniform distribution over {±1}^. 

2. For every pairwise independent distribution pL over {±1}^, there exists a degree 6n pseudo¬ 
expectation operator E over M” satisfying the constraints {x'j = l}j=i...n such that for every 
C £X and f : {±1}*^ ^ M, E/(0(x)) = E/(/i). 

The following immediate corollary implies that predicates snpporting pairwise independent dis- 
tribntions are approximation-resistant for n(n)-degree SOS: 

Corollary 1.3. For every e > 0 and P : {±1}*^ — ?• {0,1}, if there exists a pairwise independent 
distribution pL supported on P~^(l) then there exists h > 0 such that for all n there is a set X = 
{Cl,..., Cm] of k-tuples of literals over xi,... ,Xn such that 

1. For every x G {±1}", EcgxP(C(x)) < -|- e. 

2. The value of the 5n-degree Max-P SOS relaxation for the fraction of satisfiable constraints on 
the instance X is 1. 

Remark 1.4. The instance X = (Ci,.. . ,Cm) is actually obtained at random (with some pruning 
of a small fraction of the constraints, or alternatively, with some loss in the “perfect completeness” 
condition). Thus our results can also be thought as giving some evidence to a conjecture of Barak, 
Kindler and Steurer that no polynomial-time algorithm (including in particular the SOS algo¬ 
rithm) can beat the basic semidefinite program on approximating random CSP instances. 

Thronghont this paper we restrict onrselves to the Boolean case, and do not consider extensions 
to a larger alphabet, thongh onr methods may be nsefnl in this case as well. 
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1.2 Related works 

Grigoriev [15] proved in 1999 that (in the language of this paper) 3XOR is approximation resistant 
for the degree Q(n) Sum-of-Squares hierarchy. Grigoriev’s work in fact predated the papers of Par- 
rilo 120] and Lasserre m proposing the SOS hierarchy, and so he used the different (but equivalent) 
language of Positivstellensatz Galculus proofs. (Also, as far we know, he did not note that these 
proofs can be efficiently found via a semidefinite program.) Grigoriev’s result was rediscovered in 
2008 by Schoenebeck |22| . who also noted that it implies approximation resistance for 3SAT and 
some other GSPs as well. Tulsiani |25| (see also Ghan |10j l further generalized these results and in 
particular showed that every predicate that contains a pairwise independent subgroup is approxi¬ 
mation resistant for fl(n)-degree SOS. Both Tulsiani and Schoenebeck follow Grigoriev’s technique 
of reducing SOS lower bounds to resolution width lower bounds. As far as we know, no other SOS 
integrality gaps for approximating GSPs were known, and there are very few SOS lower bounds in 
general, most notably Grigoriev’s lower bound for knapsack na and the very recent result by Meka, 
Potechin and Wigderson for the planted clique problem (personal communication). 

Arora, Bollobas, Lovasz and Tourlakis [1] obtained integrality gaps for the Lovasz-Schrijver 
linear programming hierarchy for Vertex Gover. Schoenebeck, Trevisan and Tulsiani |23| showed 
that Max-Gut is approximation resistant for n(re) levels of the Lovasz-Schrijver hierarchy, and 
these results have been strengthened to the stronger Sherali-Adams hierarchy mm- The famous 
Goemans-Williamson algorithm [13] shows that Max-Gut is not approximation resistant for even 
the degree 2 SOS hierarchy, further underscoring the difference between these relaxations. 

Perhaps closest to our work are the papers of Benabbas, Georgiou, Magen, and Tulsiani [9] who 
showed that predicates containing a pairwise independent distribution are approximation resistant 
for n(n) rounds of the Sherali Adams hierarchy, even when one adds the degree 2 SOS constraints. 
Indeed, our pseudo-distribution agrees with theirs, though we describe it somewhat differently, and 
most importantly, need a completely different argument to show that it is positive semi-definite. 
Our work is also inspired by the pseudo-expectation view of the SOS hierarchy as advocated in the 
papers mm- 


2 Overview of our proof 


To prove Theorem 1.2 
{± 1 }" 


we need to show that given any pairwise independent distribution /x over 


one can come up with X, a collection of tuples {Ci,... ,Cm} of literals and a pseudo¬ 
expectation operator E that “pretends” to be the expectation of a valid distribution whose projection 
on to any Ci is /x. In fact, our choices for both X and E will not be novel and follow prior works 
in this area. For X, as mentioned, we will simply use a random set of tuples (or more accurately, 
a set corresponding to a hypergraph with sufficiently strong expansion properties), as was done 
by previous works dealing with weaker hierarchies |9l |26l [19]. It turns out that given this choice, 
the pseudo-expectation E is essentially “forced”, and again, we use the same pseudo-expectation 
used in prior works such as [9], though we describe it slightly differently. This pseudo-expectation 
corresponds in some sense to the “maximum entropy distribution” conditioned on satisfying our 
constraints (though of course it is not an actual distribution but only a pseudo-distribution in the 
sense of m)- Those prior works have shown that for every set S of o{n) variables, there is a 
distribution vs over the variables in S that agrees with E . The main difference is that we prove 
that for some d = Q{n), E is a valid degree-d pseudo-expectation operator, that is, it satisfies the 
non-negativity / positive semidefinite-ness condition E[p^] > 0 for every polynomial p < d/2. This 
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is a more “global” property, as the polynomial p might depend on all n variables, which makes it 
more challenging to prove. 

Our approach is to essentially diagonalize E. That is, we will show an explicit construction of 
polynomials xij • • ■ j XM £ 'Pd/2 we call local orthogonal functions such that (1) {xiff^i spans 

the space (^) = 0 for all i ^ j and (3) E[xf] > 0 for all i. The existence of these 

polynomials immediately implies the property we need, as, by representing every polynomial p as 
p = YliPiXi^ we see that 

° • 

i 

We now review the construction of the instance, as well as the pseudo-expectation operator, 
and then discuss how we come up with these local orthogonal functions. As mentioned above, our 
instance X = (Ci,... ,Cm) will simply be a random instance, which we think of as a A:-uniform 
hypergraph with m hyperedges Ci,... ,Cm- After some pruning we can assume this hypergraph 
has girth n(logn)j^ By a simple Chernoff + union bound argument, if m > cn for a sufficiently 
large constant c then for every assignment x G {±1}”', the induced distribution will 

be e-close to the uniform distribution. For this informal overview, suppose that we merely want to 
establish the existence of a degree d pseudo-expectation operator for some large constant d. Note 
that this means that sets of at most d (or even 2*^) variables form a forest (i.e. disjoint collection of 
trees) in this hypergraph. 

We now describe the pseudo-expectation operator E, which in some sense is almost “forced” as 
the only natural operator for this instance. (As mentioned, this part is not novel and the same 
operator was used by works such as [9]; however we describe it somewhat differently.) We construct 
E by defining for every set S of at most d variables a distribution vg over {±1}'^ such that (1) 
for every clause C contained in S, the projection of ug to C equals p and (2) the distributions 
are locally consistent in the sense that if S' C [/ then the projection of i^u to S equals vg. The 
definition of vg is very simple. First, say for the purposes of this informal overview that a set S is 
closed if every clause C in X is either completely contained in S or intersects it in at most a single 
variable. If S of size 0{d) is closed and connected (as a subgraph of X) then it is a tree in the 
hypergraph X. In this case, we define the distribution 12 g as follows: to sample x from icg we pick an 
arbitrary clause CCS and sample its variables according to p. We then continue down the tree, 
sampling the variables of all the clauses that intersect with C, and so on. It is not hard to show 
that because of pairwise independence (and in fact simply because every marginal is uniform) this 
process will always yield the same distribution regardless of the traversal order, and the probability 
of x G {±1}'^ to be sampled under this distribution will be proportional to rices~ 

If a set S is closed but not connected then the distribution ug is obtained by making independent 
choices for each of the connected components of S. For a general (not necessarily closed) set S, 
we define the closure of S, denoted by cl (S'), to be the minimal closed superset of S (this is well 
defined; one can show that intersections of closed sets are closed and thus, the minimal closed 
set is the intersection of all closed sets containing S). A fairly simple argument using the girth 
condition can be used to argue that |cl(S)| < 0(|S|) for every \S\ < d. We then define ug to be the 
distribution obtained by projecting the distribution t'ci( 5 ) to S. The collection of local distributions 

we don’t prune these clauses then our proof guarantees that for 1 — o(l) fraction of the clauses we get the 
marginal distribution to be p. It is possible that this can be upgraded to all of the clauses at the expense of some 
additional complication, but we have not checked whether or not that’s the case. 
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Figure 1: In this example, even though both A and B are collections of disjoint clauses and hence 
are “closed” under our definition, their distributions could be correlated due to the existence of the 
set C. 

so obtained satisfies (1) by construction, and it is not hard to show that it satisfies (2) as well. 
Since all polynomials of degree at most d are spanned by the set of polynomials {xs}|s|<(i (which 
we will call the characters) where Xs{x) = OieS®*’ define the pseudo-expectation operator it 
suffices to define ]E[x 5 ] for every |5| < d. We simply define Efxs] to be [xs(x)]. 

We now describe how we come up with the functions xi; • ■ ■ j XM- Intuitively, we would like 
to come up with these functions via a Gram-Schmidt like process. That is, we fix some ordering 
Ai Am of the M = (^^) sets of size at most d, and define Xi to be XA; • Now, we would 

want to define Xi to be the component orthogonal to the span of xi)---)Xi-i where we define 
orthogonality using E as an inner product. We would then get that ExiXj = 0 for all j < i, which 
would imply that Ex^Xi = 0 for all i ^ j (as Xj is spanned by xi, • ■ •, Xi)- However, this is of course 
circular reasoning, since we cannot assume that E is positive semidefinite (and hence a valid inner 
product) since this is exactly what we are trying to prove! 

However, because we know that on every small set U, E agrees with an actual expectation 
operator (the one associated with the actual distribution ujj), we do know that it is psd when it is 
restricted to this small set U. Therefore, if for some reason when we do this Gram-Schmidt process 
and express Xi some linear combination we get lucky and this linear combination 

happens to be extremely sparse then we can actually carry through the argument described above. 
Specifically, it turns out that it suffices for the set U = U{Hj | aj ^ 0} to be sufficiently small 
so that E is a valid inner product on U A Ai. However a priori, this hope seems dubious, since 
the Gram-Schmidt process is very sequential, and we need to do it for (^^) steps. It seems quite 
possible that we would create long distance correlations in the process, whereby we would end up 
needing to express Xi using many x/s for sets Aj that are quite far from H,. (See Figure]^ for one 
example of a correlation that could arise between two disjoint collection of clauses A and B.) 

Nevertheless, we show that we are in fact able to choose a tailor-made ordering of the sets so that 
this hope is (essentially) materialized. An important observation that comes to our aid here is that 
our local distributions, intuitively speaking, satisfy: if two sets A and B are sufficiently far apart in 
the hypergraph X, then the distribution vaub is obtained by taking the product of the independent 
distributions va and vb- We use this observation to argue that, if we choose the ordering on the 
sets in in the right way, then, when we express Xi as a linear combination of the functions Xj 
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Figure 2: In this example, the solid dots are variables and no clause contains any two of them, but 
the local distribution on the variables might not be uniform since the constraints of the cycle can 
create a dependency. 


for j < i, we only use j’s such that Aj is contained in a certain (carefully defined) small “ball” in 
the hypergraph around the set Ai. The crucial result that we need here is to show that whenever 
there is a dependence between the local distribution on some set A and the local distribution on 
some set B that came before A in our order, then, either B is contained in this “ball” around A, or 
the correlation between A and B is completely “explained" by the intersection of the closure of B 
with this ball, in the sense that conditioned on any assignment to the variables in the intersection, 
the local distributions on A and B are independent. This will allow us to argue that we don’t need 
to use xb to express XAi but can restrict ourselves to characters contained in that ball. Moreover, 
and crucially, we will show that our ordering has the property that all the characters we will need 
to use must have come before A as well. 

Handling Q{n) rounds. The above overview can be converted into a full proof with some care 
when d = o(log(n)) by exploiting the acyclicity of all subgraphs involved. Extending to d = 12(n) 

, however, introduces additional subtleties. When d exceeds n(log(n)), subgraphs induced by d 
vertices of I can have cycles. An immediate effect of this is that the the definition of a closed set 
that we gave before no longer yields consistent local distributions on any collection of d variables. 
An example of a problem that arises when cycles can exist on a set of vertices is illustrated in 
Figure]^ To fix this, we define a stronger notion of closed set S that guarantees that all paths of 
length at most 3 between any two vertices in S are completely contained inside S. This notion of 
closures differs from the one that Benabbas et. al. [9] use. An appeal to the expansion property 
of X (instead of high girth as before) can be used to show that the closure of a set S is at most a 
constant factor larger than [S'!. Similarly, as before, we need to show that there exists a (suitably 
defined) ball, Ball{A) around any set A of variables (of size at most d) such that the correlations 
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with any other set B of size at most d are “captured” by the intersection of Ball{A) and B. This 
needs a more careful argument. In particular, the correlations (even in the low girth case) are 
actually not necessarily captured by the intersection of BaU{A) with B, but rather with some set 
B' that is related, but not identical to B. However, the crucial property that we require is that the 
set Bin = Ball{A) n B' satishes (1) if B came before A in the ordering, then so will Bin and (2) 
\Bin\ + |i? \ Ball{A)\ < \B\. This second property is more complicated to prove in the case where 
\B\ can be much larger than the girth bound, but turns out to hold there as well. The bottom line 
is that with additional care however, the high level picture provided by this overview can indeed be 
implemented and we give a full analysis based on the local Gram-Schmidt like process in Section 

El 

3 Preliminaries 

We collect some standard dehnitions and notation here. A (k, n)-instance is a fe-uniform hypergraph 
X = {Cl,..., Cm} over [n] so that every hyperedge (also known as a clause) C = (ii ,... ,ik) € X 
is labeled by a string a = G {±1}^. We identify a clause C with the function that maps 

X G {±1}"' to yi,... ,yk where yj = ajXi-. We will sometimes also consider C as a tuple of the 
literals (cjqxq,..., We write V(C) for the variables involved in (or covered by) a clause C 

and similarly for V C [n] we write C(V) for the set of all clauses C such that V{C) C V. For any 
X G { — 1, !}”■, we write xa to denote the tuple of coordinates in the subset A C [n]. If x G {—1,1}^ 
and y G { — 1,1}^ for disjoint sets A and H, we will write x o y for the string in { — 1,that 
projects to x for coordinates in A and to y for coordinates in B. 

Unless explicitly mentioned, the base of all logarithms appearing in the paper is assumed to be 
2 . We consider the arity of our tuples fe to be a constant and so O notation may hide the dependence 
on k. 

We now dehne some standard ideas in the context of hypergraphs. 

Definition 3.1. Let G be a hypergraph. G is said to be a path if its hyperedges can be ordered 
into a sequence Ci, C 2 ,..., C^ such that for each 2 < i < £, CiCi Ci-i / 0 and Ci n Cj = 0 for every 
\i—j\ > 1. G is said to be a cycle if it has at least two hyperedges, and there is a cyclic ordering of its 
hyperedges Go, Gi,..., G^-i, and there are distinct vertices vq, ..., vi-i with u* G G* n G(j_|_i) mod e 
for all i. G is said to be a forest if it does not contain any cycle. A forest is a tree if it is connected 
(i.e. for every two distinct vertices u and v, there is a path Gi,..., G^ such that u G Gi and v G G^). 

The degree of G is the maximum number of hyperedges that intersect with any given hyperedge in 
G. The length of the shortest cycle in G is said to be the girth of G. For any vertices u,v oi a hyper¬ 
graph G, we dehne the distance, dist(tt, v) of u,v in G as the minimum number of hyperedges in any 

def 

path that joins u and v in G. For S, T, subsets of vertices, we dehne dist(S', T) = min^g^^tgr dist(s, t). 

Next, we dehne the notion of expansion in a /c-uniform hypergraph G: 

Definition 3.2 (Coefficient of Expansion). A /c-uniform constraint hypergraph G is said to be 
(r, /3)-expanding if any collection C of at most r hyperedges of G cover at least (fe — 1 — /3) |C| vertices 
of G, i.e. |{u I 3G G C, u G G}| > (/c — 1 — /3)|C|. We call /3, the coefficient of expansion of G. 

Let X be a {k, n) instance. We now describe the properties of the [k, n) instances that we need 
and give a construction for them in Section of the Appendix by taking a random instance and 
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removing a few clauses. Specifically, we show the existence of nice instances, the ones that satisfy 
the properties described in the lemma below: 


Lemma 3.3. Fix 1 > e, (5 > 0 and 7 > e^k‘^. Then, there exists a k-uniform constraint hypergraph 
G with yn edges such that for 1/r = 41og2(7A;^), G: 

1. is {r]n, 5)-expanding, 


2. has girth 9 > r log (n) 


We will use this lemma with any given e (the soundness slack), 6 = and 7 = e^/c^/e^. We 
will call the instances that satisfy the conditions of the lemma above as nice. 


For such instances, it is also easy to prove the soundness part (part (i)) of Theorem 1.2 (see 
Section B.l of the Appendix) which we record in the following lemma. 


Lemma 3.4. For every e > 0 and k, if n is sufficiently large then there exists a niee {k,n)-instance 
X with the property that for every x G {± 1 }"', the distribution {G(x)}c£i is e-close in total variation 
distance to the uniform distribution on {± 1 }^. 


4 Closed sets, and the definition of the pseudo-expectation 

Throughout the rest of this paper we hx X = (Ci,..., Gm) to be a nice {k, n) instance with coefficient 
of expansion ft. Thus whenever we mention edges, paths, or clauses, they will always be with 
respect to the hypergraph X. In this section, we dehne a linear operator E on Pf', the linear space 
of multilinear polynomials on M” of degree at most s = ^. We will ensure that the E so dehned 
will satisfy K[f{G{x))] = E[/(/i)] for every clause C G X and function / : {±1}^ ^ M. In the 
next section, we will show that the E we dehne here is in fact a pseudo-expectation operator on 
Pff for d = loooofc and thus obtain our main result. The E operator we use was dehned in previous 
works such as Benabbas et. al. |9] and later also used by Tulsiani and Worah |2^ to study weaker 
LP/SDP hierarchies. Here, we describe a construction of the same operator in a slightly different 
way so as to help us in the proof of our main result. 

To dehne E, it is enough to dehne E[xs'] for characters xs for each S' C [n], ISI < s, as one can 
then extend E linearly to all of P”. To do this, we dehne a probability distribution ux for every 
X C [n], such that |A| < s, and then set E[x 5 ] to be the expectation of xs under 1 / 5 . 

4.1 Closures 

We hrst dehne the concept of closed sets that is central to our argument. 

Definition 4.1 (Closure and closed sets). For every P > 1, a set A C [n] is R-elosed if for every 
V, v' G A, any path of length at most R between v and v' is contained in A. We say that A is closed 
if it is 3-closed. 

We dehne the R-closure of A, denoted by clij(A), to be the intersection of all sets B such that 
APB and B is P-closed. The elosure of A, denoted by cl (A), is the 3-closure of A. 





Remark 4.2. Readers familiar with the definition of closure (or advice set) in the work of or ]26f 
will find the definition of closure above slightly different. The main difference is that our definition 
allows us to have some nice properties such as uniqueness and that the intersection of two closed sets 
is closed, which are very helpful for our proof. We stress however that the actual pseudo-expectation 
is the same as that of those works. 

Next, we give a constructive definition of closure of a set. 

Lemma 4.3. Given S C [re] and any R < min{g/2, the i?-closure of S can he obtained by the 
following procedure run on S: Set A := 0. For every v,v' G U S such that there is a path 

of length at most R between v and v' in X not contained in A, add every clause in the path to A. 
Output V (^) U S. 

Proof. Observe that the procedure terminates as there are only finitely many clauses. Further, the 
output is closed by virtue of the termination of the procedure. By induction on the time at which 
a path is added in the procedure, it is easy to show that every closed set containing S must contain 
the path. Thus, V{A) is a closed set containing A and every clause C such that V{C) C V{A) 
satisfies V{C) C c\r{S). The lemma now follows by the minimality of c\r{S). □ 


Next, we bound the size of c\r{S). 

Lemma 4.4. For any R < min{g/2, 2 ^} and S C [re] such that [S'! < Then, |C(clre(S'))| < 

2R\S\ and |clre(5)| < 2Rk\S\. 


Proof. Consider the procedure described in Lemma 4.3 Let C clreC*?) be the isolated vertices 


in clre(5'). Observe that one cannot add any isolated vertices in the procedure and thus S’*®" C S. 
Define S' = S \ S'^T Then, c\it{S) = c\r{S') U 

I C/| 

If the process terminates before adding a total of g = t—^ clauses, then there’s nothing to prove, 

R~P 


since [S"! < [S’! < yields that q < '^. Thus, suppose, for the sake of a contradiction, that the 
procedure adds > q clauses and let round of the procedure be the first round where the number 
of clauses added exceeds q. 

Let Ci be the set of clauses added in the procedure till the round and let S’' be the set of 
variables obtained by taking the union of variables covered by the clauses added and S'. Further, 
suppose that the round adds qi clauses. Then, \Ci\ < q Qi < rjn and thus, Ci must satisfy 
the expansion requirement: |l/(Cj)| > (g + qi){k — 1 — /?). On the other hand, any new path of 
length j < R added in a round adds at most jk — (j — 1 ) — 2 new vertices. Thus, on an average, 
every one of the at most j new clauses added in any round of the procedure contribute at most: 
k — 1 — l/j<k — 1 — 1/R new vertices. Thus, \S'^\ < [S"! + (g + qi){k — 1 — 1/R). 

Now, 


(g + qifik - I - fi) < \ViC,)\ < |5'| < |5'| + (g + gi) • (fc - 1 - 1/R). 


This yields that |5'| > (g + g*) • (1/i? — /3) > |5'| using that g = 

The size claimed in the lemma now follows by observing that 
contributes at most k new variables. 


|S'| 

1~R 

R 


This is a contradiction. 


^ — /3 > ^ and that every clause 


2R 


□ 
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The following lemma summarizes the simple properties of the closures defined here. 

Lemma 4.5 (Simple Properties of Closures). 1. For any R < g/2, if A and B are R-closed and 
then so is An B. 

2. If A F B then cli{(A) C c\r{B). 

3. Every connected component of c\r{A) of size > 2 intersects A in at least two elements. 

4- Let A = AiU A 2 U ... Am- Then, cl(A) = cl(U”f;^cl(j4j)). 

Proof. 1. If there are two vertices v,v' in Ar\B such that dist(r;, v') < R, then since both A and 
B are closed, both of them should contain the unique (since R < g/2) path between them. 

2. By definition, clij(B) is an i?-closed set containing B D A and hence if clij(j4) ^ c\r{B) then 
clij( 74 ) n c\r{B) would be an even smaller i?-closed set that contains A, contradicting the 
minimality of clij( 74 ). 

3. Suppose otherwise that there is some connected component S of cIk(A) with [S’! > 2 inter¬ 
secting A with at most one element {x}, then we claim that B = (cl/j(4.) \ S') U {x} is an 
/Z-closed set containing A. Clearly, B D A. Now suppose for the sake of contradiction that 
there were two vertices v ^ v' of distance at most R in B whose path is not in B. Then 
since B C cl/{( 74 ) and cIr(A) is i?-closed, the path between v and v' must have had a vertex 
u G S\ {x}. But since one of v or v' must be different than x (say v'), we get by contradiction 
that v' was connected to S in cl/j(A). 

4. Let B = cl(U^]^cl(Aj)). Since cl(A) is closed and contains VJfLiAi, B C cl(^). If B ^ 

then, B D and is closed contradicting the minimality of cl (A). □ 

4.2 Definition of E 

Using the closures defined above, we define a local probability distribution on all closed sets and use 
it to define E. Let C = (xi, X 2 ,..., Vk), where, each Vj is the literal ajXi. for some aj G {±1}. The 
distribution p,c simply assigns to x G {±1}"^ the probability ;u(cJiXq,..., (JkXi^f) (i.e., the probability 
that C'(x) = a under nc is set to /i(a) for every a G {±1}^). 

The definition and the proof of consistency of the local distribution we define were shown by 
Benabbas et. al. [9| for the weaker notion of closures they used (in order to define linear round 
solutions in the Sherali Adams hierarchy). The argument for our notion of closure is similar but we 
include it here for the sake of completeness. 

For every set S' C [n], |S| < d, let cl(S) be the closure of S and suppose 1$ is the set of isolated 
variables in cl(S). Define C(cl(S)) be all clauses C such that V{C) C cl(S). Then, we set: 

J^cI(5)(2:) = ■ 2 'ci(s) • Lfc&c{c\{s))Tc{xc) ( 1 ) 

where xc the projection of x on to the coordinates in V{C), and ■^cl( 5 ) = (> 1). 

Observe that the above expression tells us that the marginal distribution of t'ci(S) over Is is uniform. 
We extend the notation above and write izr for the marginal of r'cl(T) 011 variables in T. 

We now show that r'ci(S') defined above is indeed a probability distribution over cl(S). 

Lemma 4.6. Let A and B be closed sets such that APB and \C{B)\ < rjn. Then, 
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1. I'A is a valid probability distribution: I'Aix) = 1- 

2. V is locally consistent: for every x G { — 1,1}'^, va{x) = X]j/e{-i ^b{x o y). 


The following claim that we record as a lemma will be useful in the proof. 


Lemma 4.7. There exists an ordering Ci, C 2 ,... ,Cr of clauses in Ca,b a partition of B \ A 
into sets Fi C V{Ci),F 2 C V{C 2 ),... ,Fr C V{Cr) such that for every j < r, \Fj\ > k — 2 and 

Fj n (Ui>,y(Ci)) = 0. 


We first complete the proof of Lemma 4.6 and then prove Lemma 4.7 


Proof of Lemma^ Let Za = and Zb = Let Ca,b = C{B) \ C{A). 

Using Q, we have: 


^ VB{xoy) 


^B ■ ^c&c(b)Tc{xc o yc) 

• '^C&C{A)Tc{xc) ■ '^C&CA.Bh-c{xc o yc) 


To simplify notation, we will write yn for ya x* for Xv[Ci) where x G { — 1, !}"■. We have, 
using the ordering given by Lemma 4.7 Then, 


ub{x oy) = Zb Y ■'^ceC{A)Tc{xc) ■ IIc(^Ca,b1^c{xc o Vc) 

= Zb Y ^C:^C{A)hcixc) ■ Uf^itaix^ o y^) 

Using the partition Fi, F 2 , ■ ■ ■, F^ 

= ZBUceC{A)Tcixc) ■ Y hriCrOar)--- Y MCl ° Oil) 

where Cr is the value for the variables in V{Cr) \ F^. 

Using that |Ti.| > k — 2 and pairwise independence of y 

= ZBB-C(zc(A)Tcixc) ■ Y hriCr o Q-r) ■ ■ ■ Y ^2(C2 o 0:2) ' 

Continuing similarly for 2,3,... ,r 
= ZBncec{A)Tcixc) • 2-^-1 

Now, |U(a)\i^r| = kr-\B\A\. Further, -\B\+k\C{B)\-kr + \B\A\ = -\A\+k\C{A)\. 
Thus, Zb ■ T(C’-)\U| _ completing the proof. □ 


We now complete the proof of Lemma 4.7 


Proof of LemmaFor every C G Ca,b define r(C') = {x G V{C) \ VC" / C G Ca,b, v ^ U(C')}. 
For any collection C of clauses in Ca,b, let A(C) = |Uc'ecr(C)|. Similarly, define rA(C) =r(C)\74 
and Aa(C) = | Ucgc rA(C')|. We make the following claim: 
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Claim 4.8. For any C C Ca,b, ^a{C) > {k — 5/2 — 2(j)\C\. 

We first complete the proof of the lemma using the claim. Since Aa{Ca,b) A (k —5/2 —213 )\Ca,b\ 
and /3 < 1/10, there exists a clause C such that |r^(C)| > k — 2. Now V{C) \ ^ 12 r^(C') and 
thus \V{C) \ ^1 > k — 2. We place this clause at the beginning of the ordering, call it Ci and set 
Fi = V{C) \ A. We now iterate with Ca,b \ {C} to complete the construction, obtain a clause 
C 2 £ Ca,b \ Cl such that |rn(C' 2 )| > k — 2. Since rn(C'i) cannot intersect rn(C' 2 ), we can now set 
F 2 = V{C 2 ) \ V{Ci). Continuing this way yields the required ordering and partition of i? \ ^. □ 

We now complete the proof of the claim. 


Proof of Claim /.8 Fix any C and consider any (maximally) connected subgraph with edges C C C. 
If C consists of a single clause C, then \ V{C) n 4.| < 1 (since A is closed) and V{C) n V{C') = 0 
for any C' ^ C ^C. Thus, T a{C') >k — 1. 

Now suppose C consists of at least 2 clauses. We first claim that A(C') > {k — 2 — 2j3)\C'\. To 
see this, observe that C is a collection of at most ryn clauses in X and thus, | 1 /(C^)| > (A; — 1 —/3)|C|. 
Further, every v G C(C') \ Cc^C'^{C) belongs to at least two different clauses in C and thus, 
(A: - 1 - /3)|C'| < \V{C')\ < A(C') + (A:|C'| - A(C'))/2. Rearranging gives A(C') > {k - 2 - 2P)\C'\. 

Next, we claim that that for every v G V{C') n A there exists a pair of clauses C, C such that 
V {C C C) n ^ = {n}. Consider any clause C £ C such that V{C) H ^4 = {n}. If there is another 
clause C such that V{C') Ci A = {n}, then observe that V{C') cannot intersect A in any other 
element (since A is closed) and thus we can let (7, C be the pair as above, corresponding to v. 
Otherwise, there exists a clause C such that C £ C such that V{C') H V{C) 7 ^ 0 (since V{C') is 
connected) and V(C') H A = 0 (as otherwise there would be a path between two distinct vertices 
of A, of length at most 2 outside of A). Further, observe that all such pairs are disjoint. This is 
because if some pairs intersect, then they induce a path of length at most 3 between two distinct 
vertices of A that is not contained in A (violating the 3 closedness of A). Thus, \V{C')r\A\ < |C'|/2. 
Thus, we must have: An(C') > A(C') — |C '|/2 > {k — 2 — 2/3)|C'| — |C '|/2 = {k — 5/2 — 2/5)\C'\. 

Since for every connected component C' inside C we have that Aa{C') > (k — 5/2 — 2/3)|C'|, we 
must have Aa{C) > {k — 5/2 — 2/3)|C| as promised. This completes the proof of claim. □ 


4.3 E and some basic properties 

The following is immediate from ([^: 

Lemma 4.9. Suppose A and B are closed disjoint sets such that AuB is closed. Then, i^AuBix) = 
va{xa) ■ i^b{xb)- 

We now define the pseudo-expectation operator associated with the local distributions {z^t}|t|<s' 

Definition 4.10 (Pseudo-Expectation). For the collection of consistent local probability distribu¬ 
tions {i^t}\t\<s defined in (Q for s < qn/G, we define E on P” by 

Hxs] = 


for every |S| < s. 

Corollary 4.11. Let X he a nice {k,n) instance and p, a pairwise independent distribution over 
{±1}^. Then the family of local distributions xei[n],\x\<d for s = r/n/6 satisfies: 
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Bridge Path 



Figure 3: A possible configuration when A is i2-closed and B is closed. All solid lines indicate paths 
of length at most 3. 

1. Completeness: For every clause C of I, vyic) — F- 

2. Consistency: for every S' C T C [n], \T\ < d, the marginal of vt on to S is us. 

Proof. The completeness property follows from Q and C{V{C)) = {C}. The consistency property 
follows from Lemma [4.61 □ 

Finally, since E corresponds to a valid expectation locally, we obtain that E induces a positive 
semidefinite (PSD) inner product on any space of functions of a small number of variables. 

Lemma 4.12 (Local PSDness). Let E he the pseudo-expectation operator defined by the local dis¬ 
tributions {i^s}| 5 |<s- Let T be a subset of [n] of size at most s. Then for every / G D = Spanjxyi | 
A C T}, E[/2] > 0. 

5 Local Distribution on Unions 

In this section we make an important step towards showing the positivity property of our pseudo¬ 
distribution by showing that if two sets A and B are sufficiently closed, then the local distribution 
on Ac B is only determined by the clauses that are contained in A or in B. In particular, this 
implies that if A and B are disjoint then the distribution on A is independent of the distribution of 
B. The main result of this section is the following expression for the local distribution on the union 
of A and B where A is i?-closed for a sufficiently large constant R and B is closed. 

Lemma 5.1 (Local Distribution on Union of Two Closures). Suppose A is R-closed for R > 100 
and B is closed. Then, for any x G { — 1,1}"^^^, 

^AyjB{x) = Za,b ■ '^c-.V(C)QAvjbTc{xc), 
where Za,b = 2 ^|C(^us)HAus| _ 


We make two convenient definitions before proceeding, see Figure 

Definition 5.2 (Bridge Paths). For any two closed sets A and B, a path P of length at most 3 is 
said to be a bridge path for the pair A, B A \P D A\ = \P D B\ = 1. 
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Definition 5.3 (Bridge-Closure Paths). For any two closed sets A and B, a path P of length at 
most 3 is said to be a bridge-closure path for the pair A, B, if there exists a bridge path P' such 
that IP' n PI = 1 and IP n PI = 1 but C H A = 0. 


Proof overview. Since the proof is rather technical, let us start with a high level overview of it. 
We first show the only extra clauses added to cl(AU P) come from bridge and bridge-closure paths. 
Moreover, all these additional paths are disjoint apart from their end points. What this amounts 
to is that the new connections between A and P can be thought of as a collection of disjoint trees 
Ti,... ,Tr such that each of these trees has a root in A and its leaves in P. The marginal distribution 
over A U P is obtained by summing up all possible assignments to the intermediate nodes in these 
trees. Thus at the heart of the proof is the observation that for every such tree T with root xq and 
leaves xi,... ,xi, if we consider the distribution over the variables of T induced by the tree (i.e., 
where the probability of x is proportional to nceC(T) t^c{xc)) then the marginal distribution over 
{xo, xi,..., Xf} is uniform. Hence these trees create no dependence between A and P. 

As a final remark, observe that the example from Figure shows that A and P being 2-closed 
is not enough to guarantee the statement of the lemma. While we believe that at least one of the 
sets out of A and P should be P-closed for some P > 3 for the lemma to hold, currently, we do not 
have any example of a counter example demonstrating this point. We now proceed with the actual 
proof. 


Proof of Lemma 5.1 Let D = cl(A U P). Let Ca,b and Cb be the set of bridge paths and bridge 


closure paths of P for the pair A, P, respectively. Observe that V{Ca,b) U V{Cb) T D. We now 
show that these are the only extra clauses in P: 

We first make a few simple observations: 

The first observation describes how bridge paths and bridge-closure paths intersect. 

Claim 5.4 (Intersections). 1. For any distinct Pi, P 2 £ Ca,Bj Pi O P 2 C A U P. 


2. For any distinct Pi,P2 G Cb, Pi 0 P2 C V{P) U P where P is a bridge path. 

3. For any P e Cb and P' G Ca,b, | 1 ^( P ) O V{P')\ < 1. 

4. Suppose Pi, P2 G Cb are such that Pi 0 P 7^ 0 and P2 0 P' 7^ 0 for some bridge paths P ^ P'. 
Then, Pi 0 P 2 = 0. 


Proof. 1. If the claim weren’t true, then there must be a path of length < 6 between two vertices 
of A which violating that A is P-closed. 

2. Suppose first that there is a bridge path P such that P n Pi 7 ^ 0 and P n P 2 7 ^ 0. If either 
of Pi or P 2 intersect P in more than one element, then there is a cycle of length at most 6 
in G which violates the fact that G has H(l) girth. If Pi and P 2 intersect in an element not 
contained in V{P), then, again there is a cycle of length at most 9 in G violating the high 
girth of G. Similarly, if Pi, P 2 intersect inside P, then, they cannot intersect outside of P and 
further, they cannot both intersect the same bridge path as it would yield a cycle of length at 
most 9 in G. Thus in both the cases. Pi H P 2 C V{P) U P for some bridge path P. 

3. Otherwise there is a cycle of length at most 6 in G violating that G has girth w(l). 
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4. If not, then if |P n P' Pi = 1 then there is a cycle of length 12 in the graph, contradicting 
our assumption on the girth. Otherwise \P D P' D A\ = 2 which means that there is a path of 
length at most 12 between two distinct vertices of A. 

□ 

The next observation shows that there is no path of length at most 3 that connects two bridge 
paths, two bridge-closure paths or two bridge-bridge-closure paths that are not contained in AU B. 

Claim 5.5 (No Extra Paths). 1. There is no path P of length at most 3 not eontained in A that 
eonneets a bridge path P' and A. 

2. There is no path of length at most 3 not eontained in A that eonneets P G Ca,b with P' £ Cb- 

3. There is no path of length at most 3 eonneeting distinet P,P' £ Cb- 

Proof. 1. Otherwise there is a path of length at most 6 between two vertices of A not contained 
in A, violating the fact that A is R closed. 

2. Otherwise there is a path of length at most 12 between two vertices of A, violating that A is 
R closed. 

3. Otherwise there is a path of length at most 18 not contained in A, connecting two vertices of 


The following claim is now a consequence of the claims above: 

Claim 5.6. For any C sueh that V{C) AU B but V(C) Q D, C £ Ca,b ACb- 

Proof of Claim. Consider the iterative procedure of building the closure of A U P by adding paths 
one by one in some order. Let P be the first path in this order that violates the claim. Then, either 
P intersects two bridge paths or a bridge path and ^4 or a bridge path and a bridge-closure path 
or two bridge-closure paths. Each of these situations is explicitly barred by the claims above. This 
completes the argument. □ 

Let Z = Observe that = Za,b- For every clause 

C £ Ca,bACb, let Vl. = V{C)\{ACB) and = E(C)n(GlnP). Similarly, let D' = D\{AcB) 
and D" = D H {A U B). Next, we claim: 

Claim 5.7. 

Z- E ^cgCa.buCbTc = Za,b- 

Proof. Let P' = Vi U V2 such that Vi H V2 = 0 defined by Vi = P' \ E {Ca,b)) and V 2 = D' \Vi. 
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z- ^ nceC^,BUCs^c (a^v" 0 7y^) 

= ^ X] nceCs/^c (a;y» o HceCA.sAic o 7v^) 

76{-l,l}°' 

= Z ^ nceCA.s^C (a;y- 07^^) ^ HceCsMC (a:y» o 7^^) 

76{-1,1}'^2 -^g{_17}V^l 

Now, observe that for every C G Cb, V(C) H V 2 has at most 2 elements. Thus, by pairwise independence of [i 
= Z ^ nceCA.B^c (xv^ ° 7v;t) 

7e{-i,i}'^2 

Similarly, for every C G Ca,b-, V(C) H U S) contains at most 2 elements. Thus, 

= Za,b- 

□ 

We can now write, using Q: 


^aAuB(a:) — Z ■ ^ ^c-.V{C)CDfJ'C (xy^ o 7 y^^ 

76{-l,l}^' 

Using Claim 15.61 

= -^ X/ nc..y((7)c(^uB)/^c(a:c) • nceCA.sUCfl^C (xy^ o 'jy^'^ 

7e{-l,l}a5' 

Using Claim [T7| 

= ZA,B'n.C:V(C)C{AuB)fJ‘C{xc)- 

This completes the proof. □ 


6 E is positive semidefinite 

In this section, we prove our main result. Our proof will follow easily from the following lemma 
which is the main result of this section. 

Lemma 6.1 (Main Lemma). Let = Spanjx^ | |^| < d} he the space of multilinear polynomials 
on M” of degree at most d = joq^- There exists a collection of functions {xi | 0 < i < M} C 
for M = (^^) — 1 such that: 

1. P2 = Spanjxi I 0 < z < M}. 

2 . E[x 2 ] > 0 . 

3. E[xj • Xj] = 0 whenever i ^ j. 


16 



We first complete the proof of of Theorem 1.2 assuming this lemma. Observe that part (1) of 


the theorem follows from Lemma B.3 Further, E satishes E[/(C'j)] = f{fi) by Corollary 4.11 Thus, 
we only need to prove that E is a valid pseudo-expectation operator, that is, that E is positive 
semidehnite. 

Let / G be any multilinear polynomial of degree < d. Then, we show that E[/^] > 0. We 
use the spanning property (1) of the XiS above to write / = fi' Xi- Using orthogonality (3) 

of XiS, we have: E[/^] = fi^ixf]- Finally, using the positivity property (2) of the XiS, we 

have that E is PSD. 


The rest of this section is devoted to proving Lemma 6.1 


6.1 Choosing an Ordering 

Our aim is to build an order on the (<^), in which to process them for our local orthogonalization 
procedure. We start with an arbitrary ordering on the clauses of I, e.g. for every C G X we dehne 
a unique index C(C) G [m]. We say that ^4 -< i? if: 

• C(cl(^)) is smaller than C(cl(i?)) in lexicographic order of C- That is, A B if the maximum 
index C(C) for C G 01(^4) is smaller than this maximum for cl(S), and if they are equal we 
break ties by the second largest index and so on. We dehne 7r(cl(A)) to be the index of 
cl (A) according to this ordering. (Note that vr is a permutation on distinct closures, and so if 
cl(A) 7 ^ cl(i?) then 7r(cl(^)) ^ 7r{c\{B)).) 

• If C(cl(^)) = C(cl(i?)) then we say that A < B A \A\ < \B\. 

• If C(cl(^)) = C(cl(l?)) and |^| = \B\ then we break ties arbitrarily. 

For i = 0,, M, we let Ai denote the i^^ set in this ordering. Note that Aq = % and Ai,..., An 
are the singleton elements {1},..., {n} (in some arbitrary order). We will write Xi XAi in the 
following to reduce clutter. 


6.2 Local Orthogonalization 

Set R = 100. Dehne the local correlated space as 


Vi = SpanjxB \ \B\ < d,B V c\R{Ai),B ^ Ai}. 


Lemma 6.2. For every f G V), E[/^] > 0. 


Proof. Invoking Lemma 4.12 it suffices to show that |clij(^j)| < s = r]nlQ. This follows by noting 
that \Ai\ < d and |clR(j4j)| < 2Rkd = 200r/re/10000 = rjn/500 < s (Lemma 4.4). □ 


Dehne Xi to be any f G Vi such that E[(xj — /)^] < E[(xj — g)'^] for every g G Vi. Note that 
such a function must exist because E[(xi — /)^] >0 for every / (one can WLOG minimize on the 
orthogonal complement of the kernel of E inside Vi). We dehne Xi = Xi ~ Xi- Since Vq is empty, we 
set xo ns the constant 0 function and xq is thus dehned as xo = X 0 = 1 - 
The following simple lemma would be very useful. 


Lemma 6.3 (Local orthogonality). E[xj 5 ] = 0 for every g G Vi. 
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Figure 4: A possible configuration of Bin, Bout and Bmu 


Proof. Since both g and Xi are spanned by characters of size at most d and 2d < s, the pseudo¬ 
expectation is well defined. Further, since both g and Xi Hg in Spanjxs | S C cli^(Ai)} and 
|clij(Aj)| < s (as in the proof of Lemma 6.2), E corresponds to the expectation operator associated 


with the probability distribution r'c\ii(Ai)- 

Now suppose for the sake of contradiction that 

E[(Xi - Xi)9] = <5 

for some <5 > 0. (If the expectation is negative then we can take —g.) Let f = Xi — eg. We have: 

E[(Xi - ff] = E[(X* - Xif] + e^ng^] - 2e5 
and so if e is sufficiently small then 


- /)'] < E[(Xi - Xi)'] 


contradicting our choice of x*. □ 

The following lemma shows that the xfs span 
Lemma 6.4. For every i: Spanjxj : J < i} = Spanjxj : j < i}- 

Proof. First, we show that for every i, Xi £ Spanjxj | j < i}- We argue by induction. xi = Xi nnd 
thus the statement holds for i = 1. Now suppose the statement holds for all j < i. Consider Xi- 
From the definition of Xi above, we have that: Xi = Xi + Xi- Now, Xi £Vi and Vi C Span{xj \ j < i} 
by definition. Further, by inductive hypothesis, each Xj for J < * satisfies Xj ^ Spanjxj/ | j' < j} C 
Spanjxj/ I f < i}. This completes the induction. 

The other direction is easier: Xi = Xi ~ Xi ^rid as argued above, Xi £ Span{xj \ j < *}. Thus, 
Xi G Span{xj | j < i}. Together, we thus have: Span{xj \ j <i} = Span{xj | j < i}- □ 

6.3 Global Orthogonality lemma 

In this section, we prove the following lemma that is the technical heart of the proof and says that 
local orthogonalization is enough to ensure that Xi are all mutually orthogonal. 
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Lemma 6.5. For every j < i, 


Hxi ■ Xj] = 0 . 

We will need the following observation for the proof which we record before proceeding: 


Lemma 6.6. Suppose H is a connected k-uniform hypergraph such that there exist a subset of 
vertices, U, \U\ > 2 satisfying: dist(M,r;) > R for every distinct u,v € U. Then, H must have at 
least hyperedges. 

Proof. Observe that the collection of balls of radius R/2 around any vertex in u G 17 are all disjoint 
and contain at least one path (due to connectedness of H). □ 


We now go on to prove Lemma 6.5 


Proof. Fix any j < i and let A = Ai and B = Aj. Let 


Bbdy = {x G cl(i?) I 3 a clause v{W) G C{c\{B)) s.t. W 0 cIr(74) = {x}}. 

For every x G B^dy we call any associated clause W as in the dehnition above as a boundary clause. 
Let Bqi^i — B \ cl/^(^ 2 ) and B^^i — B \ U B^dy) and By.^gf — B \ U B^yf Note that B^dy 
is not necessarily a subset of B. Next, we make two useful claims: 

Claim 6.7. 

\Bbdy U Bin\ < \B\ < d. 


Proof. We will show that \Bbdy\ < \Bout\- This immediately yields the claim by observing that 
d > \B\ = \Bin\ + \Bout\ + \Brest\ > \Bin\ + \Bbdy\- We note that the proof of this claim is 
signihcantly simpler in the case that \B\ < R/2. Proving it in the case when R is a constant and 
|i?| = n(n) is one of the main technical ingredients in getting the proof sketched in the overview to 
work for Xl{n) rounds of the SOS hierarchy. 

Let Q C [n] be a (maximally) connected component in the subgraph dehned by the hyperedges 
C(cl(.B)) \ C(clij(A)). Let Qbdy = Bbdy 0 Q and Qout = Bout O Q. Bbdy is thus partitioned into Qbdy 
for every possible maximally connected subgraphs Q. It is thus enough to prove that \Qbdy\ < \Qout\ 
for any hxed Q. 

Observe that Qncli?(^) = Qbdy If Qnclij(A) = 0, then, there is nothing to prove. If Qbdy = 
then, Q contains VfWQ where Wy is a boundary clause associated with v. If Q contains no vertex 
of Bout, then, observe that cl(i?) \ (<3\ {x}) is a closed set containing B contradicting the minimality 
of c\{B). Thus, in this case, \Qbdy\ < \Qout\- 

Now suppose for \Qbdy\ ^ 2 . Then, vertices in Qbdy are connected through clauses in Q. On the 
other hand, since A is 7?-closed, for any u,v € Qbdy, any path that uses clauses from Q between u, v 
must be of length at least R + 1. Applying Lemma 6.6 we observe that |C(Q)| > \Qbdy\B/‘^- 

Next, we claim that Q C c\{Qbdy U Qout)- It is easy to complete the proof once we have this 
claim: observe that 


\Qbdy\B/‘^ < \C{Q)\ < \C{c\{Qbdy U Qout))\ < Q\Qbdy\ + 6|Q, 


out 


Rearranging yields that \Qout\ > \Qbdy \ ' Using R > 24 yields that \Qout\ > \Qbdy\- 

We now proceed to show that Q C c\{Qbdy U Qout)- By Lemma 4.5 (4), cl(R) = c\{Bin U Bbd, 
Bout)- Let B' = Bin U Bbdy U Bout \ {Qbdy U Qout)- Then, by another application of Lemma 4.5 


U 
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(4), c\{B) = cl(cl(i?') U c\{Qbdy U Qout))- In other words, one can bnild the closnre of B by hrst 
bnilding the closnre of B' and Qbdy U Qout (Step 1 ) and then taking the closnre of the nnions of the 
obtained sets (Step 2 ). Clearly, the hnal ontpnt contains every danse in C{Q). If we show that ( 1 ) 
C{c\{B')) n C{Q) = 0 and that ( 2 ) no danse from C{Q) is added in the step 2 , then every danse 
in C(Q) mnst be added in the procednre to bnild c\{Qbdy U Qout) and thns we are done. We now 
proceed to show the two statements above. 

(1) : First observe that c\{B') itself can be bnilt by bnilding the closnre of Bin (and c\{Bin) C 
cli?(d) C{c\{Bin))r\C{Q) = 0), the closnre of Bout^ Bbdy\{Qbdy^Qout) (that cannot intersect any 
danse from C{Q) as then Q mnst inclnde a vertex from Bout U Bbdy \ {Qbdy U Qout), a contradiction) 
and hnally taking the closnre of their nnion. This last step cannot add a danse in Q: every path P 
added connects c\{Bin) and c\{Bout^Bbdy\ (Qbdy^Qout))- If P is contained in clij(d), then, there is 
nothing to prove. Otherwise P mnst pass (exactly once) throngh a bonndary vertex. If P contains 
a danse from C{Q), then, if P passes throngh a bonndary vertex not in Qbdy, then this enlarges Q 
violating that Q is a maximally connected component. If, on the other hand, P passes throngh a 
bonndary vertex in Qbdy, then, P connects Bout \ Q with Q violating the maximality of Q. Thns, 
C(cl(i3')) cannot inclnde any danse from C{Q). 

(2) : Consider the step 2 of the procednre to bnild cl(i?). In this step, we add paths (of length 
at most 3) that connect cl(i3') and c\{Qbdy U Qout)- For any snch path P, if P indndes some danse 
C from C{Q) then it crosses ont of clR(d) (exactly once) and thns mnst pass throngh a bonndary 
vertex. By maximality of Q, we mnst have that P n Bbdy £ Qbdy and P \ C(cli{(d)) C C{Q). On 
the other hand, the part of P that connects some vertex in Qbdy to c\{Qbdy U Qout) is of length at 
most 3 and thns mnst be contained in c\{Qbdy U Qout)- Thns every edge in P \ C(clij(d)) is present 
in C{c\{Qbdy U Qout) and thns C G C{Q). 

□ 


Claim 6.8. Suppose Bout 7^ 0- Then, for every S C Bin U Bbdy, S P A. 


Proof Since Bout ^ 0) cl(P) / cl(d). Thns, 7 r(cl(P)) < 7 r(cl(d)). Now, \Bin U Bbdy\ < d from 
Claim 6.7 Thns, every snbset S C Bin U Bbdy has a well-dehned ordering w.r.t (1((^). Fnrther, for 
every such S, cl(S') C cl(P) (Lemma 4.5) and thus, 7 r(cl(S')) < 7 r(cl(P)). Hence, S P A. □ 


We now proceed to comp lete the proof of the lemma. It is easy to verify that |cl/{(d)Ucl(P)| < s 


and thus by Corollary 4.11 the E operator on functions on variables in clij(d) U cl(P) corresponds 


to the expectation of a valid local distribution. In what follows, whenever we write Pr, we mean the 
probability of an event w.r.t. this local probability distribution. (Note that the expectation w.r.t. 
this probability distribution agrees with E whenever both are dehned.) 


Now, XB = XB,„XBrestXBont ^nd we can write 


HXiXj] = HXiXBir^XBrestXBo 


( 2 ) 


Consider an arbitrary assignment z to P \ d and 7 G {Tl}l^i>dHl to Let lLBhdy='y Fe the 

function that on input x G {±1}” outputs 1 if xs^^y — 7 2 :ero otherwise. 


Lemma 5.1 gives the expression for the local distribution on clij(d)Ucl(P). Using the expression, 
we have: 


^\XiXBi„XBr„stXBout I XB\A = Z, XBydy = 7 ] = ^[XiXBinXBr^st I ^ Bydy = l] ' XBo-at{z Bout), 
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where the E on the RHS matches the expectation operator associated with the probability distri¬ 
bution t'clH(A)- 

We will show that "^[XiXBinXBrest I ^B^dy = 7 ] = 0 for every choice of 7 . First, we observe that: 


• ®[XS • XBi„XBrest I ^Btdy='y] = ' XBi„XBrest ' ^Bt,dy='Y]^ 

for ever S C cIr(74), |S| < d. 

Now, Xi £ Span{x 5 | S C clR(Ai), IS"! < d}, and thus using 

Pr[lBM^=7] • HXiXBi„XBrest I XBydy = 7] = ^ [Xi ' XBi^XBrest ' '^B,dy="/]- 

Now \Bin U Bbdyl < \Bout\ (Claim [6^ and Isydy = 7 ^ Span{x5 I S C BMy}- 

XBi„XBr,st ■ ^Bydy=l ^ Span{XB„ • XBrest ' XT | 7" C Buy] ■ 


(3) 


6.3 


Each index set S of the characters above is a subset of B and thus S ^ Ai (invoking Claim 6.8 
along with the fact 7 r(cl( 5 )) < 7 r(cl(A)) ). Thus, XBi„XBrest ' ^Bydy=i ^ Using Lemma 


thus, 


HXi ■ XBi„XBrest • ^Bidy='y] “ O' 


□ 


We can now complete the proof of Lemma 6.1 


Proof of Lemma 6.1 We show that the x* constructed above satisfy all the properties required. By 
Lemma 6.4 Spanjxi | i < M} = Span{xj | i < M} = Pf. Next, observe that Xi = Xi~ Xi- Both Xi 


and Xi lie in Spanjxs | S C cli{(74j)} and by Lemma 4.12 E is a psd expectation operator over Vi. 
Thus, E[xi] > 0 for every i < M. Finally, we verify that x* mutually orthogonal. Fix any i. It is 
then eno ugh to show that E[xj • Xi] = 0 for every j ^ i. Since Spanjxr | ^ = Spanjy,. | r < j} 

(Lemma 6.4), we only need to show that E[xj • Xi] = 0 foi' every j < i. Invoking Lemma 
completes the proof. 


6.5 


then 

□ 
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A Random sparse predicates 


Consider a random sparse predicate P on k variables and accepting |P“^(1)| = t assignments. If 
t = exp(o(A:)), we now show that P does not support a pairwise independent subgroup with high 
probability, as k tends to infinity. Here the randomness corresponds to choosing P“^(l) to be a 
t-sized subset of { 0 , 1 }^ uniformly at random. 

Observation A.l. ^“^(l) does not contain any affine subspace of dimension 2 (over ¥ 2 ) with 
probability > 1 — (2^. 

Under the condition of the observation, P“^(l) does not contain any pairwise independent 
subgroup, because any such a subgroup contains an affine subspace of dimension 2 . 


Proof of Observation A.l\ Let vi,... ,Vt G P“^(l) be an enumeration of vectors in P“^(l). Note 
that if P“^(l) contains a subspace of dimension 2, then there are l<a< 6 <c<t such that this 
subspace is exactly the affine span of Va,Vb,Vc. 

For a fixed choice of the triple (a, 6 , c), conditioning on the event that Va,Vb,Vc span an affine 
subspace of dimension 2, the remaining vector from this affine subspace also belongs to P~^(l) with 
probability at most t/2^. Taking a union bound over (a, b, c) (at most t^ such choices), we see that 
P“^(l) contains an affine subspace with probability at most t^/2^. □ 
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B Constructing nice instances 


In this section, we show the existence of nice instances of constraint hypergraphs and prove Theorem 

[Til 

Lemma B.l. Fix 1 > e, <5 > 0 and 7 > e^k'^. Then, there exists a k-uniform constraint hypergraph 
G with 7 n edges such that for rj = ( 1 / 7 ^)^/^, r = G: 

1. is {rjn, 6)-expanding, 

2. has girth 9 > log (n)/r, and 

Proof. We first choose a random graph G by choosing every k uniform hyperedge, independently, 
with probability p = A'y ■ k\/n^~^. Our hnal hypergraph will be obtained by removing hyperedges 
from G. 

We hrst show that: 

Claim B.2. For G chosen as above, with probability at least 1/3, 

1. has between 2'yn and 6771 edges. 

2. has {r]n,6)-expansion, 

3. has at most n}!^ log (n) cycles of length at most g and 

We first show that the claim above is enough to complete the proof of the lemma. We dehne G' 
to be the hypergraph obtained by removing every cycle of length at most g.By the claim above, the 
total number of hyperedges removed in this process, for a large enough n, is at most yn. Observe 
that the last property in the statement of the theorem is immediately satished by G'. Further, since 
G' is obtained only by removing hyperedges from G, G' still enjoys (rjn, (5)-expansion. Thus, G' is 
a constraint hypergraph that satisfies the requirements of the lemma. Finally, the total number of 
edges removed is sublinear in n and thus G' has at least yn edges for a large enough n. 

We now move on to complete the proof of the claim above: 

Proof of Claim. 1. The expected number of edges in G is given by P ■ {^ = 47 n(l — > 

4777(1 — By an application of Chernoff bound, the probability that the number of 

edges does not lie in the interval [ 2777 , 6777 ] is at most 2 e le . 


2. Next, consider any collection of s clauses and let us compute the probability that they cover 
at most cs variables for some c = k — 1 — 6. This probability, is then upper bounded by 




P 


Using that (“) < {cs)^/kl and the approximation > 

above expression by: 



we can upper bound the 
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Using that c = k — 1 — 5 and that (5 < 1 now yields an upper bound of 


s\ 

n) 




Thus, using that 7 > and that s satisfies ^ < ( 1 / 7 ^)^/^ makes the above probability at 
most ( 1 / 7 ^)^. 


3. To see how to ensure that the high girth requirement, we first observe that for any integer i, 
the expected number of cycles of length £ in G is at most {dk'^Y. 

We first count the number of ways to choose a cycle of length i. Recall that a cycle is given 
by a cyclic sequence Ci,... ,Ci of hyperedges. There are (^) ways to choose Ci, and for 
2 < i < i, at most ways to choose the common vertex Ci-i Pi G* and remaining vertices 

for Gj, and finally at most choose G^ that intersects both Gi and G^-i. Therefore 

the expected number of length-.^ cycles is at most 




< ( 47 )'^'^ 


□ 


By an application of Markov’s inequality, with probability at least 7/8 over the draw of 
hyper edges of G, the number of cycles of length at most 9 = | n are at most 

«<g 

By a union bound, now, all the three properties above can be ensured with probability at 
least 1/3. 


□ 


B.l Soundness 

In this section, we show that after fixing the underlying hyperedges G of an instance, with high 
probability over the literals on constraints, all assignments are very close to a random assignment. 
Here closeness is measured with respect to the distribution {G(a;)} as one chooses a uniformly 
random constraint among all hyperedges of the hypergraph. 

Let G be any hypergraph with m hyperedges. Let X be an instance with the same underlying 
hypergraph as G, and with the literals in all clauses be chosen uniformly at random. We have the 
following lemma. 

Lemma B.3. Suppose m = ^(20(fc)g-2 

n). With high probability over the ehoiee of literals, for any 
assignment x G {±1}*^, the distribution {G(x)} with C ehosen uniformly at random in X is within 
e statistieal distanee to the uniform distribution over {±1}^. 

Proof. Let X = (Gi,...,Gm) be a fixed collection of literals. Let iax,x denotes the distribution 
{Gj(x)} when i is drawn uniformly from [m]. For each local assignment y G {±1}^, the probability 
Pl,x[y] that a random local assignment from yx,x equals y is given by IEjg[m][kCi(x)=?;]- 
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Now suppose the signs of the literals from X for every constraint are chosen uniformly at random, 
keeping the underlying subhypergraph fixed. Then ^x,x[y\ is now a random variable depending on 
the randomness of the literals. For each the indicator equals 1 with probability 1/2^, 

and equals 0 with the remaining probability (over the randomness of the signs of the literals on the 
i-th constraint), and the random variables lc.(^x)=y &re independent of each other for different i. 
Therefore ^Mx,x[y] is the average of m independent {0, l}-indicator random variables, each being 1 
with probability 1/2^. By Chernoff-Hoeffding bound, we have |[y] —1/2^ | > rj with probability at 
most 2 By a union bound over all assignments x G {±1}”, the maximum deviation 

of y,x,x[y] from 1/2^ (over all x) exceeds rj with probability at most 2exp(—77^m/2^^^ +nlog2). 
Letting rj = e/2^, we see that 


Pr 


max^\fix,x[y] - 1 / 2 ^ 1 } > ^ 


< exp(—n(n)) 


as long as m = n(2‘^i^ie“^n). 

Now the distribution {Ci{x)} for a random i G [m] has statistical distance at least e implies 
that ~ 1/2^1 > e/2^ for some y. By a union bound over all y G {±1}^, the distribution 

{C'j(x)} is close in statistical distance to the uniform distribution on {±1}^ except with probability 
exp(0(A:) — f2(n)), assuming m = □ 
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